Robust Entropy Rate for Uncertain Sources and its Applications in Controlling Systems Subject to Capacity Constraints

The robust entropy rate is defined as the maximum of the Shannon entropy rate, when the joint probability density function of the source is unknown. The uncertainty of the source probability density is described via a relative entropy constraint set between the uncertain source probability density and the nominal source probability density. For this class of problems, the explicit solution for the robust entropy rate is presented. Further, the results are applied to specific uncertain sources. For the fully observed uncertain Gauss Markov source, a lower bound is found for the robust entropy rate in terms of the solution of an algebraic Riccati equation of the type arising in the H∞ estimation and control. Finally, an application of the robust entropy rate for analyzing uniform asymptotic observability and stabilizability of a control/communication system is given. It is shown that for uniform asymptotic observability and stabilizability of an uncertain controlled system over an uncertain communication link, the required robust information channel capacity must be bounded below by the robust entropy rate.